Globally Optimal Cell Tracking using Integer Programming

We propose a novel approach to automatically tracking cell populations in time-lapse images. To account for cell occlusions and overlaps, we introduce a robust method that generates an over-complete set of competing detection hypotheses. We then perform detection and tracking simultaneously on these hypotheses by solving to optimality an integer program with only one type of flow variables. This eliminates the need for heuristics to handle missed detections due to occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging sequences consisting of clumped cells and show that it outperforms state-of-the-art techniques.Comment: Engin T\"uretken and Xinchao Wang contributed equally to this wor

Scalable Inference for Multi-Target Tracking of Proliferating Cells

With the continuous advancements in microscopy techniques such as improved image quality, faster acquisition and reduced photo-toxicity, the amount of data recorded in the life sciences is rapidly growing. Clearly, the size of the data renders manual analysis intractable, calling for automated cell tracking methods. Cell tracking – in contrast to other tracking scenarios – exhibits several difficulties: low signal to noise ratio in the images, high cell density and sometimes cell clusters, radical morphology changes, but most importantly cells divide – which is often the focus of the experiment. These peculiarities have been targeted by tracking-byassignment methods that first extract a set of detection hypotheses and then track those over time. Improving the general quality of these cell tracking methods is difficult, because every cell type, surrounding medium, and microscopy setting leads to recordings with specific properties and problems. This unfortunately implies that automated approaches will not become perfect any time soon but manual proof reading by experts will remain necessary for the time being. In this thesis we focus on two different aspects, firstly on scaling previous and developing new solvers to deal with longer videos and more cells, and secondly on developing a specialized pipeline for detecting and tracking tuberculosis bacteria. The most powerful tracking-by-assignment methods are formulated as probabilistic graphical models and solved as integer linear programs. Because those integer linear programs are in general NP-hard, increasing the problem size will lead to an explosion of computational cost. We begin by reformulating one of these models in terms of a constrained network flow, and show that it can be solved more efficiently. Building on the successful application of network flow algorithms in the pedestrian tracking literature, we develop a heuristic to integrate constraints – here for divisions – into such a network flow method. This allows us to obtain high quality approximations to the tracking solution while providing a polynomial runtime guarantee. Our experiments confirm this much better scaling behavior to larger problems. However, this approach is single threaded and does not utilize available resources of multi-core machines yet. To parallelize the tracking problem we present a simple yet effective way of splitting long videos into intervals that can be tracked independently, followed by a sparse global stitching step that resolves disagreements at the cuts. Going one step further, we propose a microservices based software design for ilastik that allows to distribute all required computation for segmentation, object feature extraction, object classification and tracking across the nodes of a cluster or in the cloud. Finally, we discuss the use case of detecting and tracking tuberculosis bacteria in more detail, because no satisfying automated method to this important problem existed before. One peculiarity of these elongated cells is that they build dense clusters in which it is hard to outline individuals. To cope with that we employ a tracking-by-assignment model that allows competing detection hypotheses and selects the best set of detections while considering the temporal context during tracking. To obtain these hypotheses, we develop a novel algorithm that finds diverseM- best solutions of tree-shaped graphical models by dynamic programming. First experiments with the pipeline indicate that it can greatly reduce the required amount of human intervention for analyzing tuberculosis treatment

Network Flow Integer Programming to Track Elliptical Cells in Time-Lapse Sequences

We propose a novel approach to automatically tracking elliptical cell populations in time-lapse image sequences. Given an initial segmentation, we account for partial occlusions and overlaps by generating an over-complete set of competing detection hypotheses. To this end, we fit ellipses to portions of the initial regions and build a hierarchy of ellipses, which are then treated as cell candidates. We then select temporally consistent ones by solving to optimality an integer program with only one type of flow variables. This eliminates the need for heuristics to handle missed detections due to partial occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging sequences consisting of clumped cells and show that it outperforms state-of-the-art techniques

ilastik: interactive machine learning for (bio)image analysis

We present ilastik, an easy-to-use interactive tool that brings machine-learning-based (bio)image analysis to end users without substantial computational expertise. It contains pre-defined workflows for image segmentation, object classification, counting and tracking. Users adapt the workflows to the problem at hand by interactively providing sparse training annotations for a nonlinear classifier. ilastik can process data in up to five dimensions (3D, time and number of channels). Its computational back end runs operations on-demand wherever possible, allowing for interactive prediction on data larger than RAM. Once the classifiers are trained, ilastik workflows can be applied to new data from the command line without further user interaction. We describe all ilastik workflows in detail, including three case studies and a discussion on the expected performance

Path-Tracing on a Heterogeneous Multi-GPU Cluster

Path tracing has been an offline rendering technique ever since. With the enormous increase of speed seen with today's GPU hardware, interactive generation of images with global illumination effects becomes more and more feasible. In the near future we will probably see photorealistic computer graphic rendered in real time. While image footage was formerly created on render farms, the challenge today is distributing work across a network of machines equipped with GPUs. In this work an approach to interactive global illumination is developed by assigning bidirectional path tracing workload to graphic cards in a cluster. Finally the gained performance increase is evaluated

Path-Tracing on a Heterogeneous Multi-GPU Cluster

Path tracing has been an offline rendering technique ever since. With the enormous increase of speed seen with today's GPU hardware, interactive generation of images with global illumination effects becomes more and more feasible. In the near future we will probably see photorealistic computer graphic rendered in real time. While image footage was formerly created on render farms, the challenge today is distributing work across a network of machines equipped with GPUs. In this work an approach to interactive global illumination is developed by assigning bidirectional path tracing workload to graphic cards in a cluster. Finally the gained performance increase is evaluated

Proof-reading guidance in cell tracking by sampling from tracking-by-assignment models

Automated cell tracking methods are still error-prone. On very large data sets, uncertainty measures are thus needed to guide the expert to the most ambiguous events so these can be corrected with minimal effort. We present two easy-to-use methods to sample multiple proposal solutions from a tracking-by-assignment graphical model and experimentally evaluate the benefits of the uncertainty measures derived. Ex-pert time for proof-reading is reduced greatly compared to random selection of predicted events. Index Terms — Cell tracking, uncertainty, machine learn-ing, probabilistic graphical model